Inequalities for the Bayes Risk

Several inequalities are presented which, in part, generalize inequalities by Weinstein and Weiss, giving rise to new lower bounds for the Bayes risk under squared error loss. Consider a Bayesian setting in which y denotes the observed data and θ denotes an unknown scalar parameter. Suppose that E [ (θ − E(θ|y)) |y ] and E [ (θ − E(θ|y)) ] exist. For any estimator δ(y) of θ, E [ (θ − δ(y))|y ] ≥ E [ (θ − E(θ|y))|y ] (1) E [ (θ − δ(y)) ] ≥ E [ (θ − E(θ|y)) ] (2) The right hand sides of (1) and (2) are the ultimate lower bounds as they are attainable. However, these bounds are often difficult to analyze or even to compute. Consequently, numerous lower bounds have been proposed that are presumably simpler to analyze and compute. A comprehensive survey of various approaches can be found in [1]. Starting off with the approach in [2], let ψ(y, θ) be any appropriately measurable function of y and θ. Invoking CauchySchwarz inequality, E [ (θ − E(θ|y)) ] ≥ E [(θ − E(θ|y))ψ(y, θ)] var [ψ(y, θ)] (3) provided that var [ψ(y, θ)] exists and is strictly positive. Combining (3) with (2), E [ (θ − δ(y)) ] ≥ E [(θ − E(θ|y))ψ(y, θ)] var [ψ(y, θ)] (4) Simliarly, E [ (θ − E(θ|y))|y ] ≥ E [(θ − E(θ|y))ψ(y, θ)| y] var [ψ(y, θ)|y] (5) provided that var [ψ(y, θ)|y] exists and is strictly positive. Hence, E [ (θ − E(θ|y)) ] ≥ E { E [(θ − E(θ|y))ψ(y, θ)| y] var [ψ(y, θ)|y] } (6) Combining (5) with (1), E [ (θ − δ(y))|y ] ≥ E [(θ − E(θ|y))ψ(y, θ)| y] var [ψ(y, θ)|y] (7) Combining (6) with (2), E [ (θ − δ(y)) ] ≥ E { E [(θ − E(θ|y))ψ(y, θ)| y] var [ψ(y, θ)|y] }